Optimizing Latency and Throughput Trade-offs in a Stream Processing System

The value of stream processing systems stems largely from the timeliness of the results these systems provide. Early stream processors followed the record-at-a-time approach, servicing each data point as soon as it arrives at the system. While these systems provide good latency, their behaviors become less desirable when applications require high throughput, fault tolerance, or usage of stateful or blocking operators. More recently, systems are developed to follow the micro-batch approach, where many records are processed together as small batches, providing the missing features seemingly at the cost of latency. Given the situation, we look into a micro-batch streaming system called Spark Streaming, and investigate how well the micro-batch architecture can handle latency-sensitive workloads. We instrumented Spark Streaming to understand where time goes and where development effort should be focused on. In this paper we make three contributions. First, we provide an analysis of the performance of Spark Streaming, showing the average time breakdown within the system. Second, we identify the performance and scalability bottlenecks of Spark Streaming and pinpoint the underlying deficiencies of the system. Last, we propose and evaluate several optimizations to reduce the system overhead and achieve lower latency while maintaining throughput.